Papers with deep neural network
Explanation in the Era of Large Language Models (2024.naacl-tutorials)
Copied to clipboard
| Challenge: | Explanation has long been a part of communication, where humans use language to elucidate each other and transmit information about mechanisms of events. |
| Approach: | They review the opportunities and challenges of explanations in the era of large language models and examine how they can be used to generate explanations. |
| Outcome: | The proposed methods are based on the models of large language models (LLMs) and their opaque nature. |
KT-Speech-Crawler: Automatic Dataset Construction for Speech Recognition from YouTube Videos (D18-2)
Copied to clipboard
| Challenge: | KT-Speech-Crawler is an automated dataset building tool for speech recognition. |
| Approach: | They propose an approach for automatic dataset construction for speech recognition by crawling YouTube videos. |
| Outcome: | The proposed algorithm can obtain 150 hours of transcribed speech in a day with an estimated 3.5% word error rate. |
LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation (2020.acl-demos)
Copied to clipboard
Dong-Ho Lee, Rahul Khanna, Bill Yuchen Lin, Seyeon Lee, Qinyuan Ye, Elizabeth Boschee, Leonardo Neves, Xiang Ren
| Challenge: | Existing frameworks for sequence labeling and classification require massive human effort and labeling data is limited. |
| Approach: | They propose a web-based, Label-Efficient AnnotatioN framework that allows an annotator to provide the needed labels for a task and can capture explanations for each labeling decision. |
| Outcome: | The proposed framework surpasses baseline F1 scores by 5-10 percentage points while using 2X times fewer labeled instances. |
Post-Specialisation: Retrofitting Vectors of Words Unseen in Lexical Resources (N18-1)
Copied to clipboard
| Challenge: | Word vector specialisation is a portable, light-weight approach to fine-tuning distributional word vector spaces by injecting external knowledge from rich lexical resources such as WordNet. |
| Approach: | They propose a constraint-driven vector space specialisation method that embeds external knowledge into lexical resources into a deep neural network to specialise unseen words. |
| Outcome: | The proposed method preserves useful linguistic knowledge for seen words while propagating external signal to unseen words to improve their vector representations. |
Locally Distributed Activation Vectors for Guided Feature Attribution (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to explain predictions of deep neural networks are unstable and do not always provide faithful explanations to the target model. |
| Approach: | They propose a method to learn explanations-specific representations while constructing deep network models for text classification. |
| Outcome: | The proposed method improves model interpretability while preserving predictive performance. |
Bridging Languages through Images with Deep Partial Canonical Correlation Analysis (P18-1)
Copied to clipboard
| Challenge: | a deep neural network can be used to improve bilingual text embeddings . a novel approach is proposed to optimize text embed-ings on shared visual information . |
| Approach: | They propose a deep neural network that leverages images to improve bilingual text embeddings. |
| Outcome: | The proposed model outperforms previous methods on word similarity and cross-lingual image description retrieval. |
A Multi-sentiment-resource Enhanced Attention Network for Sentiment Classification (P18-2)
Copied to clipboard
| Challenge: | Existing sentiment classification approaches do not fully exploit sentiment linguistic knowledge. |
| Approach: | They propose a Multi-sentiment-resource Enhanced Attention Network to integrate sentiment linguistic knowledge into the deep neural network via attention mechanisms. |
| Outcome: | The proposed network captures sentiments from different representation sub-spaces, and is superior to strong competitors. |
A New Concept of Deep Reinforcement Learning based Augmented General Tagging System (C18-1)
Copied to clipboard
| Challenge: | Existing systems for general sequence tagging/labeling are based on neural network architectures. |
| Approach: | They propose a deep neural network based sequence labeling model and a augmented tagger to improve system performance by modeling the data with minority tags. |
| Outcome: | The proposed system outperforms the current state-of-the-art model on ATIS and CoNLL-2003 datasets by 1.9% and 1.4%. |
Learning Disentangled Textual Representations via Statistical Measures of Similarity (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to disentangle a sensitive attribute from textual representations require training and multiple parameter updates. |
| Approach: | They propose a family of regularizers for learning disentangled representations that do not require training. |
| Outcome: | The proposed regularizers are faster and faster and achieve better results when combined with pretrained and randomly initialized text encoders. |
Extracting Chemical-Protein Interactions via Calibrated Deep Neural Network and Self-training (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Several natural language processing methods have been used to extract interactions between chemicals and proteins from biomedical text data. |
| Approach: | They propose a method to extract chemical–protein interactions from biomedical text data . they use a pre-trained language-understanding model and calibration techniques to estimate uncertainty . |
| Outcome: | The proposed approach achieves state-of-the-art performance on the Biocreative VI ChemProt task while preserving higher calibration abilities. |
Deep Probabilistic Logic: A Unifying Framework for Indirect Supervision (D18-1)
Copied to clipboard
| Challenge: | Indirect supervision is a promising direction to address the annotation bottleneck . end-to-end modeling with probabilistic logic is often intractable due to inference and learning . |
| Approach: | They propose a framework for indirect supervision that integrates deep learning with deep learning by combining probabilistic logic with deep-learning. |
| Outcome: | Experiments on biomedical machine reading demonstrate the potential of this framework. |
Automatic Transcription Challenges for Inuktitut, a Low-Resource Polysynthetic Language (2020.lrec-1)
Copied to clipboard
| Challenge: | Inuktitut is one of the 60 Indigenous languages currently spoken in Canada . polysynthetic languages are often termed agglutinative when their morphemes have clear boundaries and thus are easily segmentable. |
| Approach: | They propose to use a corpus of 23 hours of transcribed oral stories to train automatic speech recognition in Inuktitut. |
| Outcome: | The proposed model shows that Inuktitut displays a much higher degree of polysynthesis than other agglutinative languages like Finnish or Turkish. |
Accuracy meets Diversity in a News Recommender System (2022.coling-1)
Copied to clipboard
| Challenge: | Existing news recommender systems use news stories that users have read in the past to infer their interests and preferences. |
| Approach: | They propose a two-tower architecture that learns news representation through a news item tower and users’ representations through s query towers. |
| Outcome: | The proposed architecture achieves a balance between accuracy and diversity on two news datasets. |
Domain Knowledge Empowered Structured Neural Net for End-to-End Event Temporal Relation Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to extract event temporal relations from text data are limited by hard constraints and large datasets. |
| Approach: | They propose a framework that enhances deep neural network with distributional constraints constructed by probabilistic domain knowledge to improve the baseline neural network models. |
| Outcome: | The proposed framework improves baseline models with strong statistical significance on two widely used datasets in news and clinical domains. |
Incorporating LIWC in Neural Networks to Improve Human Trait and Behavior Analysis in Low Resource Scenarios (2022.lrec-1)
Copied to clipboard
| Challenge: | Psycholinguistic knowledge resources have been widely used in constructing features for text-based human trait and behavior analysis. |
| Approach: | They propose to incorporate a widely-used psycholinguistic lexicon into NN models to improve human trait and behavior analysis in low resource scenarios. |
| Outcome: | The proposed methods perform significantly better than baselines that use only LIWC or NN-based feature learning methods. |
Constructing Word-Context-Coupled Space Aligned with Associative Knowledge Relations for Interpretable Language Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to train language models have limitations in interpretability . a Word-Context-Coupled Space (W2CSpace) is proposed to improve the performance of pre-trained models . |
| Approach: | They propose a Word-Context-Coupled Space to replace pre-trained models with interpretable statistical logic. |
| Outcome: | The proposed language model can achieve better performance and highly credible interpretability compared to state-of-the-art methods. |
Linguistically-Informed Self-Attention for Semantic Role Labeling (D18-1)
Copied to clipboard
| Challenge: | Existing models of semantic role labeling use no explicit linguistic features. prior work has shown that syntax trees can dramatically improve SRL decoding. |
| Approach: | They propose a neural network model that incorporates syntax using only raw tokens . they show that LISA out-performs the state-of-the-art with contextually-encoded word representations a 1.0 F1 on newswire and 2.0 F1 in out-of domain text . |
| Outcome: | The proposed model outperforms the state-of-the-art model with word embeddings and predicted predicates. |
HS-GC: Holistic Semantic Embedding and Global Contrast for Effective Text Clustering (2024.lrec-main)
Copied to clipboard
| Challenge: | In this paper, we introduce Holistic Semantic Embedding and Global Contrast (HS-GC) to learn the instance- and cluster-level representations. |
| Approach: | They propose a novel loss function that exploits different layers of semantic information in a deep neural network to provide a more holistic semantic text representation. |
| Outcome: | The proposed model outperforms the state-of-the-art model on five text datasets and improves clustering accuracy of 5.9% and 3.2% on the StackOverflow and TREC datasets. |
A Multimodal German Dataset for Automatic Lip Reading Systems and Transfer Learning (2022.lrec-1)
Copied to clipboard
| Challenge: | Lip reading is a visual observation of a speaker's lips that can be used for communication problems. |
| Approach: | They present a dataset of 250,000 publicly available videos of speakers of the Hessian Parliament which was processed for word-level lip reading using an automatic pipeline. |
| Outcome: | The proposed dataset GLips (German Lips) is compared with the LRW dataset and shows that it has language-independent features. |
Improving Recall of Large Language Models: A Model Collaboration Approach for Relational Triple Extraction (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing large language models can extract triples from simple sentences with few-shot learning or fine-tuning, but they often miss out when extracting from complex sentences. |
| Approach: | They propose an evaluation-filtering framework that integrates large language models with small models for relational triple extraction tasks. |
| Outcome: | The proposed framework integrates large language models with small models for relational triple extraction tasks. |
DNN-based Speech Synthesis Using Abundant Tags of Spontaneous Speech Corpus (2020.lrec-1)
Copied to clipboard
Yuki Yamashita, Tomoki Koriyama, Yuki Saito, Shinnosuke Takamichi, Yusuke Ijima, Ryo Masumura, Hiroshi Saruwatari
| Challenge: | Experimental evaluation results show that rich annotations enhance the reproducibility of paralinguistic features of synthetic speech. |
| Approach: | They investigate the effectiveness of using rich annotations in deep neural network-based statistical speech synthesis. |
| Outcome: | The proposed method improves reproducibility of paralinguistic features of synthetic speech . the corpus of spontaneous Japanese (CSJ) has large annotations on paralinguistic and nonlinguistic features . |
A State-Vector Framework for Dataset Effects (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent DNN-based systems gain linguistic abilities on multiple levels ranging from syntax, semantics, and even some discourse-related abilities. |
| Approach: | They propose a state-vector framework that uses idealized probing test results as the bases of a vector space to quantify the effects of both standalone and interacting datasets. |
| Outcome: | The proposed framework allows to quantify the effects of both standalone and interacting datasets. |
RENN: A Rule Embedding Enhanced Neural Network Framework for Temporal Knowledge Graph Completion (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for temporal knowledge graph embedding do not account for structural dependencies between relations. |
| Approach: | They propose a framework that enhances temporal knowledge graph completion through rule embedding. |
| Outcome: | The proposed framework improves temporal knowledge graph completion through rule embedding. |